AUTHORS: Marwa Hussien Mohamed, Mohamed Helmy Khafagy, Mohamed Hasan Ibrahim

Download as PDF

ABSTRACT: Map-Reduce are a programming model which is widely used to extract valuable information from enormous volumes of data. Map-reduce designed to support heterogeneous datasets. Apache Hadoop mapreduce used extensively to uncover hidden pattern like, data mining, SQL, etc. The most important operation for data analysis is joining operation. But, map-reduce framework doesn’t directly support join algorithm. This paper explain and compare two- way and multi- way map-reduce join algorithms for map reduce also we implement MR join Algorithms and show the performance of each phase in MR join Algorithms. Our experimental results show that map side join and map merge join in two-way join algorithms has longest time according to preprocessing step sorting data and reduce side cascade join has the longest time at Multi-Way join algorithms

KEYWORDS: - Hadoop, Map-Reduce, Multi-Way Join, Two-Way Join, Ubuntu

REFERENCES:

[1] Dean, J., and Ghemawat, S.: ‘MapReduce: simplified data processing on large clusters’, Communications of the ACM, 2008, 51, (1), pp. 107-113

[2] Kyong-Ha Lee, H.C., Bongki Moon: ‘Parallel Data Processing with MapReduce: A Survey’. Proc. SIGMODDecember 2011 pp. Pages

[3] Pigul, A.: ‘Comparative Study Parallel Join Algorithms for MapReduce environment’2013 pp. Pages

[4] VIKAS JADHAV1, J.A., SUNIL DORWANI2: ‘JOIN ALGORITHMS USING MAPREDUCE: A SURVEY’, International Conference on Electrical Engineering and Computer Science, 21- April-2013

[5] Ebada Sarhan, Atif Ghalwash,Mohamed Khafagy,Agent-Based Replication for Scaling Back-end Databases of Dynamic Content Web Sites”,ICCOMP'08 Proceedings of the 12th WSEAS international conference on Computers

[6] Ebada Sarhan, Atif Ghalwash,Mohamed Khafagy ,Queue Weighting Load-Balancing Technique for Database Replication in Dynamic Content Web Sites ',APPLIED COMPUTER SCIENCE (ACS'09) University of Genova, Genova, Italy, 2009, Pages 50-55

[7] Ahmed M Wahdan Hesham, A. Hefny, Mohamed Helmy Khafagy,” Comparative Study Load Balance Algorithms for Map Reduce Environment” International Journal of Applied Information Systems,2014, Issues 7(11),pp 41-50.

[8] V.VIJAYALAKSHMI, A.A., S.NAGADIVYA: ‘THE SURVEY ON MAPREDUCE’, V.Vijayalakshmi et al./ International Journal of Engineering Science and Technology (IJEST), 07 July 2012, Vol. 4 (0975-5462), pp.1-8

[9] Palla, K.: ‘A comparative analysis of join algorithms using the Hadoop map/reduce framework’, Master of science thesis. School of informatics, University of Edinburgh, 2009

[10] Blanas, S., Patel, J.M., Ercegovac, V., Rao, J., Shekita, E.J., and Tian, Y.: ‘A comparison of join algorithms for log processing in MapReduce’. Proc. Proceedings of the 2010 ACM SIGMOD International Conference on Management of data 2010 pp. Pages

[11] Chandar, J.: ‘Join Algorithms using Map/Reduce’, Magisterarb. University of Edinburgh, 2010

[12] Khafagy, M.H. ; Feel, H.T.A.,Distributed Ontology Cloud Storage System” IEEE,Proceedings of the 2012 Second Symposium on Network Cloud Computing and Applications Pages48-52

[13] al Feel, H.T. ; Khafagy, M.H.OCSS: Ontology Cloud Storage System”,IEEE Network Cloud Computing and Applications (NCCA), 2011 First International Symposium on Pages 9-13

[14] Haytham Al Feel, Mohamed Khafagy, Search content via Cloud Storage System. International Journal of Computer Science Issues (IJCSI)bVolume 8 Issue 6, 2011

[15] Min Chen, S. Mao, Y. Liu, 'Big Data: A Survey', ACM/Springer Mobile Networks and Applications (ACM MONET) , Vol. 19, No. 2, pp. 171-209, April 2014

[16] Lee, T., Kim, K., and Kim, H.-J.: ‘Join Processing Using Bloom Filter in MapReduce’. Proc. Proceedings of the 2012 ACM Research in Applied Computation Symposium2012 pp. Pages

[17] Jiang, D., Tung, A., and Chen, G.: ‘Map-joinreduce: Toward scalable and efficient data analysis on large clusters’, Knowledge and Data Engineering, IEEE Transactions on, 2011, 23, (9), pp.

[18] Lei Liu, D.L., 1 Shuai Lü,1,2 and Peng Zhang1: ‘An Abstract DescriptionMethod of Map-ReduceMerge Using Haskell’, 1 College of Computer Science and Technology, Jilin University, Changchun 130012, China College ofMathematics, Jilin University, Changchun 130012, China17 July 2013 pp.

[19] Mina Samir Shenouda, Mohamed Helmy Khafagy, Samah Ahmed Senbel,” JOMR: Multijoin Optimizer Technique to Enhance MapReduce Job”, The 9th International Conference on INFOrmatics and Systems (INFOS2014),2014 pp 80-86

[20] YongChul Kwon, M.B., Bill Howe,Jerome Rolia: ‘A Study of Skew in MapReduce Applications’2012 pp.

[21] www.tpc.org

[22] Ebada Sarhan, Atif Ghalwash, Mohamed Khafagy, Specification and implementation of dynamic web site benchmark in telecommunication area, Proceedings of the 12th WSEAS international conference on Computers 2008 Pages 863-86